A Comparison of the Coefficient of Predictive Power, the Coefficient of Determination and AIC for Linear Regression

The coefficient of determination, R, is widely used as a measure of fit for model selection and variable selection. However, R does not offer any insight about the quality of the predicted values or the potential influence of particular observed values on the predicted values. In regression analysis, statistical procedures such as t test, F test, and prediction intervals provide insight into the quality and the potential influence of individual observations on the estimate. The prediction sum of squares (PRESS statistic) is used as an indication of the predictive power of a model. Computing the PRESS statistic consists on fitting the model, repeatedly, leaving out an observation each time. In each repetition the model is used to predict the observation that was left out. PRESS simulates prediction by leaving out the observation that it is trying to predict. An external residual for the i observation is equivalent to calculating the external predicted value without the use of the i ˆ Y i ( ) th observation. Since is not used in fitting the regression model,